Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rate limit middleware to support long latency requests #10737

Closed
2 tasks done
inforly opened this issue May 18, 2024 · 1 comment
Closed
2 tasks done

rate limit middleware to support long latency requests #10737

inforly opened this issue May 18, 2024 · 1 comment

Comments

@inforly
Copy link

inforly commented May 18, 2024

Welcome!

  • Yes, I've searched similar issues on GitHub and didn't find any.
  • Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?

rate limit middle ware can't work well with long latency requests, it should be due to the TTL. Since LLM (large language model) usually take long time (several to tens of seconds), it really needs rate limit to support such long latency requests. Please see the details of the issue: #10700

@inforly inforly changed the title rate limit support long latency requests rate limit middleware to support long latency requests May 18, 2024
@jspdown
Copy link
Contributor

jspdown commented May 23, 2024

Duplicate of #10700

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants